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This study introduces three growth modeling techniques: latent growth modeling (LGM), hierarchical linear 
modeling (HLM), and longitudinal profile analysis via multidimensional scaling (LPAMS). It compares the 
multilevel growth parameter estimates and potential predictor effects obtained using LGM, HLM, and LPAMS. The 
purpose of this multilevel growth analysis is to alert applied researchers to selected analytical issues that are required 
for consideration in decisions to apply one of these three approaches to longitudinal academic achievement studies. 
The results indicated that there were no significant distinctions on either mean growth parameter estimates or on the 
effects of potential predictors to growth factors at both the student and school levels. However, the study also 
produced equivocal findings on the statistical testing of variance and covariance growth parameter estimates. Other 
practical issues pertaining to the three growth modeling methods are also discussed. 
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Longitudinal studies provide important sources of 
information when investigating how differences in various 
national and regional school policies, practices and 
compositional characteristics relate to differences in student 
achievement over a period of time. Therefore, it is not 
surprising that the use of growth modeling techniques in 
educational fields has rapidly increased. Recent years have 
produced a vast range of applications of structural equation 
modeling based (SEM) latent growth modeling (LGM) in 
applied longitudinal data analysis (Curran & Hussong, 2002; 
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Meredith & Tisak, 1990; Muthen, 1991; Willett & Sayer, 
1994). With the flexible usage of latent growth factors and 
measurement error structures, LGM is capable of testing the 
relative fit of various competing models. Within the domain 
of hierarchical structure, the hierarchical linear model (HLM) 
has created a powerful set of techniques for research on 
individual change (Bryk & Raudenbush, 1987; Foorman, 
Francis, Novy, & Liberman, 1991; Huttenlocher, Haight, 
Bryk, & Seltzer, 1991; Rogosa & Willet, 1985). When applied 
with valid measurements from a multiple-time-point design, 
this model affords an integrated approach for studying the 
structure and predictors of individual growth (Raudenbush & 
Byrk, 2002). Both techniques have the ability to test models 
that include multiple levels of hierarchical structured data and 
the capacity to embed assessments in more complex models 
that assess potential predictors, mediators, and consequences 
of change (Curran & Hussong, 2002; Raudenbush & Byrk, 
2002). Additionally, such approaches are better able to assess 
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the multivariate patterning change across multiple measures, 
compared to other traditional methods (Willett & Sayer, 
1996). The differences are that in LGM, each participant is 
required to have the same number and spacing of time points, 
and level- 1 predictors with random effects must have the same 
distribution across all participants in each subpopulation, 
while HLM allows unequal numbers and spacing of time 
points as well as the different distributions of level- 1 
predictors (Raudenbush & Bryk, 2002). 

However, the underlying assumptions therein would 
appear to limit their applicability. LGM and HLM are based 
on a likelihood-based approach in computing population 
parameters. This maximum likelihood (ML) technique^ is 
developed under the assumption of multivariate normality and 
requires a large sample size (Bollen, 1989; Curran, 2003; 
Jdreskog, 1969). A large sample yields a covariance matrix 
that is able to produce a better estimate of the population 
covariance matrix, and should therefore be expected to offer 
enhanced opportunities to successfully identify the correct 
model (MacCallum, 1986). Moreover, under the assumption 
of a multivariate normal distribution of the observed variables, 
ML estimators have the desirable asymptotic properties of 
being unbiased, consistent, and efficient (Kmenta, 1971). 
Unfortunately, there were many situations in which these 
assumptions were violated in practice. For instance, Micceri 
(1989), in his survey of empirical data sets, found that the 
distributional characteristics of 440 large-sample achievement 
and psychometric measures were all significantly nonnormal. 
Despite the development of robust approaches, Chou, Bender, 
and Satorra (1991) and Gold, Bender, and Kim (2003) argued 
that improvements in Yuan-Bend er ML and asymptotic 
distribution free method (ADF) are clearly needed, in 
particular under extreme nonnormal conditions and/or small 
sample size. Many researchers (Boosma, 1985; Anderson & 
Gerbing, 1984; Enders & Bandalos, 2001) have pointed out 
that sample sizes of 100 or less tended to result in high rates 
of non-convergence in SEM analysis. Therefore, the 
likelihood estimators may not have the desirable properties. In 
addition, a covariance structure approach has proven 
problematic when intercorrelations among some variables are 
high. Kline (1998) suggested that multicolhnearity is the 
major reason why a sample covariance matrix may be non- 
positive, and that certain mathematical operations are either 
impossible, or the results are unstable, because some 
denominators are very close to zero. Thus, the covariance 
matrix cannot be inverted to compute the parameter estimates, 
and it may yield larger variances and covariances of parameter 


estimates which in turn can affect significance tests (Biesanz, 
Deeb-Sossa, Papadakis, Bollen, & Curran, 2004; Brekke, 
Long, Nesbitt, & Sobel, 1997; Hudenlocher, Haight, Bryk, & 
Seltzer, 1991; Smith, Landry, & Swank, 2000; Stoolmiller, 
1995). 

In recent years, some pioneers (Davison, Gasser, & Ding, 
1996; Davison, Kang, & Kim, 1999; Ding, Davison, & 
Petersen, 2005; Kim, Frisby, & Davison, 2004) have 
demonstrated that concepts of individual growth modeling can 
be accommodated within the framework of multidimensional 
scaling named longitudinal profile analysis via multidimensional 
scaling (LPAMS). The strength of LPAMS is to allow for the 
simultaneous estimation of intra- and inter-individual growth 
processes using a relatively small number of statistical 
assumptions. Since LPAMS is similar to a hierarchical 
modeling in the sense that it allows one to estimate both 
overall (level-2) and individual (level- 1) growth rates, further 
considerations in interpreting the test results are not required. 
Additionally, it does not require a large sample size or 
multivariate normality, but assumes only the homogeneity of 
variance across time. As with any exploratory approach, it is 
designed to identify patterns of growth underlying a set of 
data, not to test a priori hypotheses regarding patterns (Ding, 
2003). Therefore, it is considered most useful in the early 
stages of a research program in which little is understood 
about the underlying growth patterns. Lastly, because LPAMS 
is not based on covariance structure but on proximity distance 
measure (e.g., squared Euclidean distance), multicolhnearity 
does not cause any computational problems within LPAMS 
technique. 

The purpose of this study is to investigate the differences 
and similarities and to discuss the strengths and weaknesses of 
LGM, HLM, and LPAMS. In so doing, this study compares 
multilevel growth parameter estimates, model-fit indices, and 
potential predictor effects in answering four central research 
questions: 1) What students’ mathematical growth trajectories 
should be expected, 2) How academic growth rates differ 
regarding potential predictors (English language program 
learner; ELP, special educational program; SEP, and gender), 
3) In multilevel analysis, what achievement patterns found at 
the school level should be expected, and 4) How growth 
trajectories are influenced by a school level predictor (school 
location: urban and suburban schools). Through four primary 
analytical approaches, this study aims to alert applied 
researchers to selected analytical issues that are required for 
consideration in terms of the decision to apply one of these 
approaches to academic growth analysis. 
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Growth Modeling Techniques 

Latent Growth Modeling (LGM) 

Latent growth modeling (LGM) pertains specifieally to 
latent growth factors, rather than to observed repeated 
measures of a construct over time. LGM attempts to smooth 
over observed measures in order to estimate the continuous 


trajectory that gives rise to these time specific observed 
measures (Curran & Hussong, 2002). Therefore, we use the 
observed repeated measures to estimate latent growth factors, 
thereafter focusing on analyzing these latent growth factors. 
To estimate these growth factors, the conceptual models are 
written as the following formations of matrices. Suppose that 
with a linear growth structure one has four repeated measures. 
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where i indicates a specific individual and n notes total number of subjects. 
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The first row of the left side matrix in equation 1 
indicates that the first subject {i = 1) is repeatedly measured 
four times. Thus, n subjects have been collected across four 
time points. In LGM formula, observed repeated measures can 
divide into two parts that are on the one hand true latent 
growth function and on the other error term. If a linear model 
is imposed, true latent growth is obtained through multiplying 
two latent growth factors (e.g., rj .^ : the individual intercept 
latent factor, 7 ] -2 : the individual linear growth latent factor) 
and two factor loadings (a row of I’s for intercept factor and 


the coding of time: 0, 1, 2, t - 1 for linear factor). From matrix 
equation 2, the estimates of the mean intercept and growth 
factors (//q[ and // 02 ) computed. These are referred to 
as the ‘unconditional latent growth curve model.’ When an 
exogenous variable (X) is presented so as to predict observed 
variability, the equation will be changed. In this case, the 
measurement equation remains the same, but the structural 
formula is extended to include the effects of these predictor 
variables. 
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Regression eoeffieients 
to intereept and linear growth faetors 
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This model is referred to as the ‘eonditional growth eurve 
model.’ For the purposes of this researeh, these eonditional 
and uneonditional LGM equations are used to estimate the 
growth faetors, and test the relative fit of the eompeting 
models. 

In edueation, studying growth patterns of standardized 
test seores has been explored in order to investigate the effeets 
of improvement efforts on aeademie performanee. These 
efforts are often evaluated in terms of the growth effeets of 
within-level (e.g., individual and student level) growth and in 
terms of between-level (e.g., teaeher, elass, sehool, and sehool 
distriet level) growth^. Therefore, poliey makers and sehool 
administrators are able to obtain valuable information through 
a multilevel analysis in aeademie growth studies. For instanee, 
multilevel models are used to answer whether the growth rates 
of eertain sehools are faster than those of others. It also 
explores how aggregated level faetors (e.g., teaeher 
qualifieation, sehool budget, sehool size, ete) affeet between- 
level growth rates. A number of researehers (Bentler & Liang, 
2003; Linda, Lee, & Poon, 1993; Longford & Muthen, 1992; 
Muthen, 1989, 1994) suggested that this multilevel strategy 
allows for the disaggregation of the within (e.g., student level) 
and between (e.g., sehool level) eovarianee stmetures within a 
single partitioned eovarianee matrix. This is then used as the 
unit of analysis in the estimation. SEM software (e.g. EQS, 
LISREL, M-plus) is used to eompute between-level and 
within-level eovarianee matriees and between-level means 
from a raw data file, as well as the eurrent iteration estimates 
of model parameters. 

Hierarchical Linear Modeling (HLM) 

Raudenbush and Bryk (2002) noted that many individual 
ehange phenomena are able to be represented through a two- 
level hierarehieal level. At level- 1, eaeh person’s development 


is represented by an individual growth trajeetory that is 
dependent on a unique set of parameters. They added that 
these individual growth parameters beeome the outeome 
variables in a level-2 model. Thus, under a linear model at 
level- 1 and level-2, equations 4 and 5 simplify to 

Yu = ^,1 + (4) 

= Pm + ra (5) 

= Pm +ni 

In this regression equation, y.^ is an observed test seore 
for person i at time t and a,_^ is the ending of time variable 
(e.g., 0, 1, 2, ..., t - 1 for a linear growth stmeture). and 
7T^2 ^re respeetively the individual intereepts and the 
regression eoeffieients with representing the residual 
error term. When one predietor variable (X) is presented, 

= Pm + 

7Tj2 = Po2 4" ^i'2^ 4- T-2 (6) 

The level-3 model represents the variability in an 
aggregated variable (k). We ean view the level-2 means, 
and y 0 o 2 jt ’ varying randomly around a grand mean: 

Poik ~/oii Po2k ~ y <m +^02/t C^) 

This equation ean also be extended for the level-3 
eonditional model, as per equation 6. 

The method for formulating the model in HLM is similar 
to that in the LGM approaeh. In equation 1, the individual 
growth faetors ( and t ]-2 ) are equivalent to a matrix 
formation of and 71^2 in equation 4. and P ^2 
indieate respeetively the average of intereept and slope growth 
faetors the same as and //qj of LGM. 
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However, the estimation methods of LGM and HLM are 
fundamentally different. Curran (2003) explained that the 
assumption of independenee of observations is highlighted in 
the standard estimation of the SEM in that the diserepaney 
funetion is based on a single aggregate sample eovarianee 
matrix that allows for the eovarianee strueture within any 
other level of nesting that is assumed to be null. In eontrast, 
the estimation of the HLM outeome ineorporates eomplex 
data struetures among lower and higher levels of data 
hierarehy in that the nesting in the data is explieitly modeled. 
Thus, Curran (2003) added that whereas nested data struetures 
pose a signifieant problem to standard ML estimation in SEM, 
the estimation of the HLM explieitly allows for these 
dependent struetures. Moreover, time seores are treated as 
data in HLM whereas time seores of LGM are treated as 
parameters. 

Longitudinal Profile Analysis via Multidimensional Scaling 
(LPAMS) 

The longitudinal profde analysis via multidimensional 
sealing (LPAMS) starts its analysis from the following 
equation: 

yi,=Ci+Y,w.x,+s,, ( 8 ) 

where jp,, is an observed test seore for person i at time 
t, Cj is a / X 1 veetor of eonstant intereept terms, W, is a 
profde mateh index eharaeterizing person i, and is a 
growth seale value refleeting the loeation of a repeated 
variable at time t (Kim, Frisby, & Davison, 2004). If A: (> 2) 
growth dimensions are required (i.e., multidimensional growth 
eases), there will be k numbers of 

and X, (x,j X ^2 )• LPAMS eonsists of three steps 

(Ding, 2003). The analysis begins with a matrix eontaining a 
proximity measure defined over all possible pairs of variables. 
In this study, the variables are time points, and the proximity 
measured for eaeh possible pair of time points, a squared 
Euelidean distanee measure"^, is eomputed from the raw data. 
When proximity measures are submitted to an appropriate 
multidimensional sealing algorithm, the analysis is expeeted 
to yield one dimension for eaeh growth eurve. Sinee the 
objeetive of LPAMS model analysis is to determine whether 
there are partieular trend shapes found within the data, and 
whether those shapes are linear eurves, nonlinear eurves, or 
time-series periodie eurves, a multidimensional sealing 
(MDS) estimation method would be eapable of identifying 
one or more sueh trend shapes where sueh eurves exist in the 


data. 

In the seeond step, the zero point of the seale values are 
reset so that the seale values indieate growth rates for eaeh 
time-point, and in order that the intereept estimate ean be 
interpreted as the initial growth level. Ding, Davison, and 
Petersen (2005) and Kim, Frisby, and Davison (2004) have 
explained that if the zero point is set to eorrespond with the 

growth seale value as at the first time -period, then that will 

* 

result in X[ =0 for all profile k. Therefore, the growth seale 

value of the first time -point is reset to zero on eaeh profile in 

* 

sueh a way that X^ = — Xj . The resealed growth seale 

values estimates would have a range of zero to some positive 
or negative numbers, depending on the shape of the eurve. 
These growth seale values would represent the growth rate for 
eaeh time -point. The authors added that c, beeomes the 
expeeted seore under the model for person i at initial time t = 
1, refleeting the initial growth level. In the third step, the 
intereept ( C - ) and profile mateh index parameters ( W - ) ean 

be estimated for eaeh individual by regressing the observed 

* 

seores ( jp,, ) onto the resealed value estimates ( X, ). 

These LPAMS mathematieal terms ean be interpreted as 
per these of LGM. The averages of C, and are 
respeetively equivalent to //gj and jU q 2 indieating the 
means of intereept and growth faetors in LGM. Also, X^ 
ean be viewed as a speeial ease of faetor loading (A^, ) in 
LGM (see equation 1). The differenee is that LPAMS direetly 
ealeulates X^ from the data (random effeet), whereas LGM 
eonsiders a fixed effeet (e.g. 0, 1, 2 ,..., t - 1 for the linear 
growth model). Table 1 presents an analogy of the elements 
and notations of three growth modeling struetures. 

Sinee LPAMS offers parameter estimates and a fit index 
of individual growth, it appears to be useful for applied 
researehers interested in predieting eaeh subjeet’s initial seore 
and growth rate. On the other hands, LGM provides average 
growth information and an overall fit index. One other 
differenee is that the LPAMS model estimates within 
individual ehanges with respeet to the latent ehange eurves 
using the MDS method and estimates between-individual 
variations via a eonventional analysis of varianee (ANOVA) 
approaeh, while HLM represents a elustering of individuals 
within group, and variables are measured at all available 
levels. This model, then, eombines variables from different 
levels in one statistieal model (Ding, 2003). In addition, the 
LPAMS model is based on a distanee model (Borg & 
Groenen, 1997; Davison, 1983) rather than a linear model. 
Thus, it ean be used to model data that, by nature, are 
nonlinear. For eonditional analysis, LPAMS is easily eapable 
of testing the effeets of potential predietors to growth profiles 
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Figure 7. 15 Randomly Selected Individual Growth Curves for Mathematics Achievement Scores over Four Time Points 


267 



Tacksoo Shin 


by using the ANOVA approach. LPAMS performs 
aggregation of the individual test results into high level 
variable (aggregated variable) for multilevel analysis. 

Methodology 

Sample Description 

The data set was obtained from a sample of 1244 students 
from a large school district. These students were in the 2"“^ 
grade at the first instance of measurement, and were then 
followed for four years (from 1998 to 2001). The students took 
a mathematics achievement test, (the Northwest Achievement 
Level Math Test), with vertically equated scale scores each year 
from the 2"^ to 5* grades. Both LGM and LPAMS models 
require time-structured data so that all sample members were 
observed on the same set of occasions. Figure 1 displays the 
math growth trajectories of 15 randomly chosen students. From 
the graph of their individual curves, we might infer that most 
individual math scores tended to increase over time. 

Three covariates measured for student level analysis at 
the first time-point were considered. These potential 
predictors of change were coded: 0 = students were not in 
special educational program (SEP), and 1 = students were in 
SEP; 0 = female, and 1 = male; 0 = non-English language 
leaner (ELL), and 1 = ELL. 53% of the students were male. 
12% were in special educational programs and 24% had 
limited English proficiency levels. The students were enrolled 
in a total of 28 schools. The ranges of the number of students 
nested within one school were 30 through 68. 7 schools were 
situated in suburban area, with this school location considered 
an exogenous variable for school level analysis. 

Results 

Table 2 summarizes the unconditional test results of 
student and school levels for LGM, FILM, and LPAMS 
models. The quadratic growth models of LGM and FILM 
yielded the best fit in terms of student and school level. A 
LGM’s standard decrement-to-chi-squared test between linear 
and nonlinear models (e.g., student level: change in = 
13.370 on df^ 3 with the critical value 7.815 at a = .05) 
and HLM’s deviance test between two models (e.g., student 
level: change in deviance = 40.862 on df= 5 with the critical 
value 11.070 at a = .05) also revealed that the quadratic 
model fit the data better than the linear model. Flowever, the 


means and variances of quadratic factors for both student and 
school levels were not significant. Additionally, the linear 
growth model with heterogeneous error variance structure 
presented reasonable model-fit statistics. Thus, this study 
interprets the change and variability of math achievement with 
a linear growth function. As previously mentioned, where 
there is more than one growth trend (e.g., nonlinear curve) in 
the data, the LPAMS model can simultaneously identify these 
curves with single growth factor. Therefore, the model-fit 
testing between linear and nonlinear models is not necessary 
under LPAMS analysis conditions. 

Unconditional Student and School Level Analyses 

For the unconditional student level analysis, all methods 
yielded similar mean growth parameter estimates. The 
average initial and growth scores of LGM, FILM, and LPAMS 
were respectively 194.580, 194.532, and 194.286, and 8.800, 
8.975, and 9.824. Flowever, the predicted mean scores for 
each year should be calculated differently. In this study, while 
the growth factor loading is fixed, such as at 0, 1, 2, 3 for a 
linear model in LGM and FILM, the LPAMS model uses a 
sample-driven factor loading, such as at 0, .992, 1.757, 2.723. 
Again, the factor loading (scale value) has a random effect in 
LPAMS, whereas time is entered as the values of the factor 
loadings relating the repeated measures to the underlying 
latent growth factors in LGM and FILM. The growth levels of 
15 randomly selected individual students under LPAMS are 
presented in Table 3. 

The variance components of initial and growth parameter 
estimates for all three approaches were statistically 
significant. This suggests that there was meaningful evidence 
of inter-individual heterogeneity at the first time-point and 
over time. Flowever, LGM and FILM tended to produce the 
smaller intercept and slope variance estimates (168.503 and 
4.744 for LGM; 169.303 and 4.446 for HLM) than LPAMS 
(194.652 and 12.728). Significantly, the major difference was 
found in covariance estimate testing. The covariance 
parameter estimate found under LGM and FILM (respectively 
10.512 and 10.571) revealed a significant positive relationship 
between intercept and growth factors (correlation = .372 for 
LGM and .385 for FILM). This estimate suggests that growth 
rates of students reporting higher math scores in the first year 
would be relatively faster than those of students reporting 
lower achievement scores in the same year. In contrast, the 
covariance of LPAMS ( .083) was not found to be significant. 
This result indicates that students’ initial scores would not be 
related to their academic growth rate levels. 
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Table 2 


Results of Unconditional Multilevel Analyses 



Student Level 

School Level 


LGM 

Heterogeneity 

. LPAMS 

Heterogeneity 

LGM LGM 

Heterogeneity Homogeneity 

HLM 

Homogeneity 

LPAMS 

Average 

° 1 Q4 ssn*** 

Intercept 

194.532** 194.286*** 

194.138*** 194.195*** 

194.195*** 

193.893*** 

Average 

6 8.800*** 
Slope 

8.795*** 9.824*** 

g 794*** g 7Q7*** 

8.701*** 

9 725*** 

Variance of 

1 08 s^O*** 

Intercept ios.dju 

169.303*** 194.652*** 

24.445*** 27.354*** 

29.305*** 

34.917*** 

Covariance 





ofintercept 10 512 

and Slope ^ 372 )** 

10.571 (.385)** .083 (.002) 

2.805 (.625)* 2.235 (.528) 

1.435 (.245) 

1.602 (.114) 

Variance of , 

Slope 4.744** 

4.446** 12.728** 

.825 .957 

1.168* 

1.753* 

= 52.126 

Deviance = STRESS 

;if^= 19.466 ;if^ = 24.921 

Deviance = 

STRESS 

Model with df=5 

35449.761 <.001, 

with (7/=8 With df=\ 1 

35264.986 

<.001, 

Fit 7 »-value = .000, 

Average 

7 >-value = .013, 7»-value=.009, 


Average 

RMSEA=.064 

R-square 

RMSEA=.045 RMSEA=.048 


R-square = 


= .865 



.857 

*** p < .001; ** p < .01; * p < .05. 





Note. The parenthesis ( ) indicates the correlation values. 




Note. Heterogeneity refers to a heterogeneous error structure being imposed. Homogeneity indicates that the homogeneous error variance 

is implemented. 





Table 3 





Growth Information of 15 Randomly Selected Students by LPAMS 




Student ID 

Intercept ( C, ) 

Growth Profile ( ) 

Model-Fit (r-square) 

464480 

199.87 

6.02 

0.81 


701170 

174.96 

7.41 

0.86 


513985 

169.18 

8.75 

0.96 


546458 

188.64 

5.73 

0.93 


637986 

185.19 

12.76 

0.97 


748844 

178.41 

6.75 

0.92 


546325 

182.21 

11.75 

0.94 


837838 

175.55 

11.35 

0.98 


337442 

174.16 

10.3 

0.88 


917907 

186.28 

7.33 

0.91 


925041 

185.07 

13.93 

0.95 


338119 

195.95 

13.54 

0.94 


688223 

176.05 

11.09 

0.88 


294498 

172.36 

14.21 

0.89 


836006 

187.79 

9.84 

0.95 



269 



Tacksoo Shin 


The different model-fit indiees were eonsidered for the 
three modeling teehniques. In LGM, most applied researehers 
are interested in the ehi-square fit test and the root mean 
square error of approximation (RMSEA). Various fit indiees 
(e.g., GFI, AGFI, NFI, CFI, ete.) are also supplied by SEM- 
based software. HEM yields the devianee values and finds the 
most plausible model through the devianee-differenee test. 
For LPAMS, the adequaey of one-dimensional MDS solution 
is verified by the MDS fit index, STRESS formula 1 (Kmskal, 
1964). Furthermore, an r-square value is imposed on the 
model so as to examine the goodness of fit. In this study, the 
linear LGM exhibited a reasonable fit. The ehi-square fit 
statistie was 52.126 with df ^ 5, p < .001. With over 1000 
people in the sample, this ehi-square statistie has a large 
degree of power to deteet even small deviations from the 
model. The GFI (goodness of fit index) equaled .995. The 
RMSEA equaled .064. Aeeording to the guidelines of Browne 
and Cudeek (1993), an RMSEA of .05 or less signifies a elose 
fit, with that at or below .08 indieates a reasonably fitting 
model. While there were statistieally signifieant deviations 
from the model, these deviations were small, and the 
eovarianee stmeture showed a reasonably good level of fit. In 
FILM, the linear model with a heterogeneous error stmeture 

model fit the data better than that with the homogeneous error 

2 

varianee model (devianee of differeneex = 28.465 on df^ 3 
with a eritieal value 7.815 at a = .05). Lastly, the STRESS 
formula 1 value of LPAMS was less than .001, indieating the 
rank ordering of four proximity data points eould be perfeetly 


reprodueed by the one -dimensional solution obtained. The 
mean r-square value of the LPAMS approaeh was .865. This 
indieated that the average 86.5 % of total variability was 
aeeounted for by the LPAMS growth equations. 

All three modeling teehniques again yielded similar 
mean growth parameter estimates at the sehool level. These 
results indieated that sehools were reporting signifieant initial 
math seores of approximately 194 points, and linear rates of 
inerease around 9 points per time-point. Flowever, dissimilar 
results of statistieal testing on varianee and eovarianee 
parameter estimates were diseovered. The varianee estimates 
of LPAMS were larger than those of LGM and HLM. 
Moreover, the varianee of LGM growth rates (.825) was not 
signifieant, while that of FILM and LPAMS growth profiles 
(respeetively 1.168 and 1.753) expressed meaningful evidenee 
of sehool variability over time. In the ease of the eovarianee 
parameter estimate, LGM found a signifieant positive 
relationship between initial and trae growth faetors 
(eovarianee = 2.805, eorrelation = .625). This estimate 
suggests that the sehools that reported higher math seores at 
the beginning time -point tended also to report faster rates of 
inerease than sehools reporting lower math seores in the first 
year. On the other hands, FILM and LPAMS revealed no 
signifieant relationship between intereept and growth faetors. 

Conditional Student and School Level Analyses 

The test results of the three different approaehes suggest 


Table 4 


Results of Conditional Multilevel Analyses 



Student Level 



School Level 



LGM 

Heterogeneity 

HLM 

Heterogeneity 

LPAMS 


LGM 

Heterogeneity 

HLM 

Homogeneity 

LPAMS 

Intercept and LLP 

-2.129 (-.377)** 

-11.614** 

-11.633** 

Intercept and 
LOCATION 

1.619 (.538)** 

8.845** 

9.197** 

Intercept and SEP 

-2.503 (-.272)** 

-10.822* 

-10.789** 


- 

- 

- 

Intercept and 
GENDER 

1.621 (.198)* 

1.514* 

1.506* 


- 

- 

- 

Slope and LLP 

-.027 (-.029) 

-.070 

-.006 

Slope and 
LOCATION 

.126 (.134) 

.614 

.448 

Slope and SEP 

-1.005 (-.204)* 

-1.873* 

-2.078* 


- 

- 

- 

Slope and 
GENDER 

.069 (.063) 

.258 

.289 


- 

- 

- 


***/»< .001; **/» < .01; */» < .05. 

Note. The parenthesis ( ) indicates the correlation values. 
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that a moderate level of eollinearity exists between the 
eovariates and the growth faetors of math seores. LGM found 
that with a negative relationship between English language 
learner (ELL) status and initial faetor (eovarianee estimate = - 
2.129; eorrelation estimate = - .377), we are able to eonelude 
that students who were in ELL reported lower math seores 
than students not in said program at the beginning time -point. 
With respeet to the speeial edueation program (SEP), effeets 
were found on both the intereept and growth terms. The 
negative relationship between SEP and the initial faetor 
(eovarianee = - 2.503; eorrelation = -.272) indieated that SEP 
students reported lower math seores in the 2"‘* grade. 
Additionally, the growth rate of SEP students (eovarianee = - 
1.005; eorrelation = - .204) was relatively lower than that 
found among non-SEP students. Finally, male students 
reported higher math seores in the first year (eovarianee 
=1.621; eorrelation = .198), while there was no signifieant 
gender-related impaet on the growth rates. HEM and LPAMS 
also showed that students who took the English language 
program reported lower math aehievement levels, to the extent 
of an average of 11.614 (11.633 for LPAMS) points in the 
first year. SEP students reported lower initial and growth 
seores of respeetively 10.822 and 1.873 (10.789 and 2.078 for 
LPAMS) points than non-SEP students. Finally, male students 
tended to obtain slightly higher average math seores at the 
beginning time -point (i.e., as mueh as 1.514 points for FILM, 
and 1.506 pints for LPAMS). 

There was one potential predietor for the eonditional 
analysis at the sehool level, whieh was sehool loeation (7 
suburban and 21 urban sehools). By LGM, the results of 
testing the sehool loeation variable suggested that sehool 
loeation had a signifieant effeet on the intereept faetor, 
whereas growth rate was not influ eneed by the same variable. 
This result indieates that sehools loeated in suburban areas 
reported higher math seores than urban-loeated sehools at the 
beginning time -point (eovarianee = 1.619, eorrelation = .538). 
HEM also revealed that suburban sehools reported higher 
math seore, by as mueh as 8.845 (9.197 for LPAMS) points in 
the first year, with no subsequent relationship between sehool 
loeation and growth rate. Table 4 illustrates the test results of 
eonditional student and sehool levels. 

Discussion 

LGM, HEM, and LPAMS methods of analysis resulted 
in no signifieant differenees in average growth parameter 
estimates in the student and sehool level analyses, regardless 


of the imposition of different growth factor loadings. In the 
case of the effects of potential predictors to growth factors, all 
modeling techniques yielded the same test results. The math 
achievements of non-SEP, non-ELL, and male students tended 
to be higher than those of other students at the beginning time- 
point. The growth rate of non-SEP students was also faster 
than that of SEP students. For the purposes of the multilevel 
analysis, although urban schools had a tendency to report 
lower academic achievements, there were no significant 
differences in the growth rates depending on school location. 

However, this study presented some dissimilar test 
statistics among LGM, HEM, and LPAMS methods. Firstly, 
the student and school levels’ variance parameter estimates of 
LGM and HEM tended to be smaller than those of LPAMS. In 
the case of school level, HEM and LPAMS produced 
significant school level differences with regard to growth 
rates, whereas LGM did not. Importantly, the major 
discordance among growth techniques was the significance 
test results on covariance parameter estimates. LGM 
persistently showed that initial status and true gains were 
positively related in student and school level analyses. This 
indicates that these achievement gaps increased, as students 
and schools with lower scores at the beginning also reported 
lower growth rates. The patterns of variances for student and 
school levels over time fully support these interpretations. 
From 1998 to 2001, since the variances of student and school 
levels tended to increase over time (respectively, 214.564 > 
209.987 < 259.662 < 294.077; 40.368 > 34.078 < 51.294 < 
57.035), we should therefore conclude that the achievement 
gaps among students and schools widen. HEM showed a 
significant relationship between intercept and slope in the 
student level, while that relationship disappeared in the school 
level analysis. Additionally, none of the covariance estimates 
in student and school levels were deemed significant under the 
LPAMS approach. This suggests that the achievement gaps 
among students and schools were parallel across years. 

There are a number of potential reasons behind 
dissimilarities in test results among LGM, HEM, and LPAMS 
methods. Firstly, LGM and HEM use the ML technique to 
estimate the population parameter estimates, whereas the 
ordinary least squares estimator is imposed to obtain LPAMS 
parameters. When observations are independent of one 
another and are normally distributed with a constant variance, 
both ML and least square methods intersect (Myung, 2003). 
However, most research situations do not meet these 
conditions. Especially under longitudinal study, since it tracks 
the same persons and involves observations of repeated 
measures over time meaning that auto-correlated errors are an 
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important consideration, ML estimates are more likely to 
differ from ordiamy least square estimates. Secondly, the 
assumption of homogeneity of variance would affect the test 
statistics. Although LPAMS assumes a homogeneous error 
variance only, an equal variance assumption is more likely to 
violate this in actual longitudinal analysis. In this study, given 
that the homogeneity of the variance test was rejected, we 
should be careful when interpreting the test results produced 
by LPAMS. The level-3 (between-level) growth information 
would also be influenced by this assumption. Raudenbush 
and Bryk (2002) argued that when unidentified slope 
heterogeneity at lower-level appears as the heterogeneity of 
lower-level error variance, such slope heterogeneity might be 
expected to bias estimates of the higher-level coefficients. 
Related to this issue, the different measurement error 
structures may be sufficient to cause different statistical 
testing on variance and covariance estimates. LGM can 
impose both the heterogeneous and homogeneous error 
variance models on the within- (level-2 in HLM) and 
between-level (level-3 in HLM) testing, while HLM cannot 
implement the heterogeneous error variance model on the 
analysis of level-3. Therefore, where the homogeneous error 
structure is implemented within an LGM context at the school 
level, the variance of the slope factor tended to be larger, and 
covariance between the initial status and true gain became 
insignificant, as illustrated in Table 3. 

Another possible explanation of the differences in the 
test results may be due to sample size. LGM and HLM 
procedures, which are maximum likelihood based approaches, 
would not ordinarily produce stable results if the sample size 
at any level is too small. Additionally, given a small country- 
level sample size (27 countries), Cheung and Au (2005) 
reported that results at the individual level were quite stable 
even when using such small individual-level sample sizes, 
whereas the group-level parameter estimates and their 
standard errors were unsystematically affected by varying 
individual-level sample sizes. In this study, the total number 
of schools was 28 and the average number of students within 
the school level was approximately 41. The major concern 
here is the lack of a clear cut point determining how many 
samples need to be collected at any given level, and how 
many lower-level data are required within higher level studies. 
Although LPAMS has no serious limitation in relation to 
sample size, regression and ANOVA approaches under 
LPAMS analysis also require a large enough sample size to 
achieve the desired power level of significance testing. If not, 
there is no way to guarantee the stable result. Therefore, one 
may not be certain as to whether the estimator is estimating a 


meaningful value. Lastly, high correlations between measured 
variables may yield problematic test results on parameter 
estimates of LGM and HLM methods. This multicollinearity 
would cause inflated standard error estimates leading to 
inflating the type II error rate, especially for covariance 
structure analysis. Vasu and Elmore (1975) indicate that 
violation of the assumption of normality coupled with the 
condition of multicollinearity results in large standard errors 
in the sampling distributions of the standardized coefficients. 
According to the results of Grewal, Cote, and Baumgartner 
(2004), when multicollinearity is extreme, type II error rates 
can be unacceptably high, being greater than 80% in cases 
where the correlations are greater than .80. In this study, the 
ranges of correlations in student and school levels were 
respectively .82 to .91 and .88 to .97. Under these data 
abnormalities, LGM and HLM may be differentially sensitive. 
Fortunately, it may be anticipated that the assumption of 
multivariate normality did not greatly affect the test results. 
The ranges of univariate skewness and kurtosis of variables 
were respectively .043 to .122 and -.045 to -.207, with the 
value of Mardia-based Kappa (mulitivariate normality test) 
tending to zero (. 102) 

From a statistical point of view, these results make it 
difficult to select a single most appropriate modeling 
technique. Although applicability of LGM would be limited 
under several statistical conditions (e.g., multicollinearity, 
small sample size, multivariate noimormality), it has a more 
flexible array of possible covariance structures for modeling 
random effects and residuals (Rovine & Molenaar, 1998). 
Additionally, LGM is able to model and comparatively 
evaluate a broader array of growth functions (du Toit & 
Cudeck, 2001). HLM is less likely to be associated with 
estimation problems (i.e., difficulty in obtaining convergence 
of the estimation procedure) and efficiently handles the 
unbalanced hierarchical data structure. However, this 
hierarchical modeling technique also shares most of the 
statistical limitations of LGM. LPAMS is an integrated 
technique for exploring developmental growth trends (e.g., 
systematic and directional growth) as well as exploring 
change patterns (e.g., oscillation between ups and downs) that 
are not growth in nature (Ding, Davison, & Petersen, 2005). 
In particular, if a study includes many time points (e.g., more 
than 7 time points) and more than two growth dimensions 
require change detection, LPAMS must be a useful technique. 
Thus, it is widely applicable to various longitudinal studies 
including academic achievement growth and psychological 
change (e.g., mood change per day). However, as discussed 
above, the homogeneous error variance assumption may be 
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easily violated in real eireumstanee and the violation of this 
assumption threatens the validity of statistieal inferenees. 

Lastly, several other researeh eonditions eould limit the 
findings of this study being able to be more widely 
generalized. First, the results of this analysis may not be 
applieable when growth trend shows a nonlinear ehange. 
Seeond, though the repeated measured variables tend to be 
highly eorrelated, the eross-time eorrelation pattern in this 
study is extraordinary high (.80 to .90) for mueh edueational 
and psyehologieal data. Unlike LPAMS, LGM and FILM 
funetion oddly at this level of abnormality of data. Thus, if 
lower eorrelations between variables are obtained, test results 
may be different. Related to this issue, further longitudinal 
study should examine the unique and eombined effeets of 
varying sample sizes at within (e.g., student level) and 
between (e.g., sehool level) measures and modeling diverse 
growth stmetures, as well as a ehanging degrees of 
nonnormality and of eorrelations between observed variables 
for eaeh growth method. 

Conclusion 

This study suggests that applied researehers should seleet 
appropriate growth modeling teehniques, depending on their 
researeh questions and eireumstanees. If one is interested in 
eaeh individual growth rate, LPAMS is appropriate. In 
addition, sinee LPAMS does require a small number of 
statistieal assumptions, it would be able to provide reasonable 
test results under a diverse range of adverse researeh 
eonditions (e.g., multieollinearity, multivariate nonnormality, 
ete). Being flexible and effieient in terms of teehnieal and 
struetural aspeets, LGM is deemed preferable for large sample 
based multilevel studies (e.g., national level studies) and for 
longitudinal researeh involving heterogeneity of error 
varianee aeross time points. FILM would be superior to other 
growth teehniques, in partieular in unbalaneed time-spaeing 
data strueture and where dependeney among observations is 
observed. Signifieantly, when the modeling teehniques yield 
equivoeal results, espeeially in terms of statistieal testing of 
varianee and eovarianee estimates due to the imposition of 
differing statistieal estimation teehniques and assumptions, 
researehers should be eareful when interpreting the inter- and 
intra-variability levels and the relationship between initial 
seores and true gains. The reeommendation herein is to 
examine whether the varianees of measurements ehange over 
time (e.g. inerease and/or deerease), or remain parallel. If the 
trends of varianees deerease over time, this may indieate that 


individual variability has deereased, and that the initial seores 
tended to be negatively related to growth rates. On the other 
hand, where varianees inerease over time, the individual 
differenees may appear to have inereased, and the relationship 
between initial and growth faetors have exhibited the 
tendeney to be positively related. While researehers remain in 
need of performing statistieal tests of eovarianee parameter 
estimate, this deseriptive information is neeessary to 
understand the general trends in the data, as well as offering 
helpful adviee in determining the most appropriate growth 
parameter estimates. 


Notes 

1. This work was supported by BK21 Academic Leadership 
Institute for Competency-based Education Reform, 
Department of Education at Seoul National University. 

2. Simply, maximum likelihood technique is to find a parameter 
{6) that maximizes the sample likelihood (i.e., make the 
observed data most likely). Suppose that we collected data 
(measurements y) from the population. Probability of y given 
0 is normalized probability density (| p^y | 0^dy = 1) of y 
as a function of parameter 0 , then the parameter 0 can be 
estimated by joint probability density for the n measurements 

L{y 1^) = n p(y> 1^) 

This L{y\0) — ,y„ \0) is called likelihood 

function. Maximizing L by varying 0 amounts to 
interpreting L as function of 0 , given the measurements y. 

3. Compared with the hierarchical Linear Modeling (HLM), the 
within-level analysis is referred to as level-1 and level-2. The 
between-level analysis is indicated as level-3. 

4. Euclidean distance is to measure the ordinary distance 
between two points. In longitudinal study, these points 
become time points, and the proximity measure for each 
possible pair of time points (f, t ) is a squared proximity 

distance measure, 0^^: , computed from the raw data as 
follows; 

Thus, in the matrix of squared Euclidean distance, all main 
diagonal values should be equal to zero and off diagonal 
values indicates the distances among time points. In the case of 
covariance matrix, the main diagonal and off diagonal values 
respectively notes variance and covariance. 


273 



Tacksoo Shin 


References 

Anderson, J. C., & Gerbing, D. W. (1984). The effeet of 
sampling error on eonvergenee, improper solutions, and 
goodness-of-fit indiees for maximum likelihood 
eonfirmatory faetor analysis. Psychometrika, 49, 155- 
173. 

Bender, P. M., & Liang, J. (2003). Two-level mean and 
eovarianee struetures: Maximum likelihood via an EM 
algorithm. In S. P. Reise & N. Duan (Eds.), Multilevel 
modeling: Methodological advances, issues, and 

applications (pp. 53-70). Mahwah, NJ: Lawrenee 
Erlbaum Assoeiates, Ine. 

Biesanz, J. C., Deeb-Sossa, N., Papadakis, A., Bollen, K. A., 
& Curran, P. J. (2004). The role of eoding time in 
estimation and interpreting growth eurve models. 
Psychological Methods, 9, 30-52. 

Bollen, K. A. (1989). Structural equations with latent 
variables. New York: John Wiley & Sons. 

Boosma, A. (1985). Noneonvergenee, improper solutions, and 
starting values in LISREL maximum likelihood 
estimation, Psychometrika, 50, 229-242. 

Borg, I., & Groenen, P. (1997). Modern multidimensional 
scaling: Theory and applications. New York: Springer. 

Brekke, J. S., Long, J. D., Nesbitt, N., & Sobel, E. (1997). The 
impaet of serviee eharaeteristies on funetional outeomes 
from eommunity support programs for persons with 
sehizophrenia: a growth eurve analysis. Journal of 
Consulting and Clinical Psychology, 65, 464-475. 

Browne, M. W., & Cudeek, R. (1993). Alternative ways of 
assessing model fit. In K. A. Bollen, & J. S. Long (Eds.), 
Testing Structural Equation Models (pp. 136-162). 
Beverly Hills, CA: Sage. 

Bryk, A. S., & Raudenbush, S. W. (1987). Applieation of 
hierarehieal linear models to assessing ehange. 
Psychological Bulletin, 101, 147- 158. 

Chou, C. P., Bender, P. M., & Satorra, A. (1991). Sealed test 
statisties and robust standard for non-normal data in 
eovarianee straeture analysis: A Monte Carlo study. 
British Journal of Mathematical and Statistical 
Psychology, 44, 359 - 368. 

Cheung, M. W.-L., & An, K. (2005). Applieations of 
multilevel struetural equation modeling to eross-eulture 
researeh. Structural Equation Modeling, 12, 598 - 619. 

Curran, P.J. (2003). Have multilevel models been struetural 
equation models all along? Multivariate Behavioral 
Research, 38, 529-569. 

Curran, P. J., & Hussong, A. M. (2002). Struetural Equation 


Modeling of Repeated Measures data: Latent Curve 
Analysis. In D. S. Moskowitz & Seott L. Hershberger 
(Eds.), Modeling Intraindividual Variability with 
Repeated Measures Data: Methods and Application. 
New Jersey: Lawrenee Erlbaum Assoeiation. 

Davison, M. L. (1983). Multidimensional scaling. New York: 
Wiley. 

Davison, M. L., Gasser, M., & Ding, S. (1996). Identifying 
major profile patterns in population: An exploratory 
study of WAIS and GATB patterns. Psychological 
Assessment, (S, 26 - 31. 

Davison, M. L., Kang, H., & Kim, S. (1999). The strueture of 
ability profile pattern: A multidimensional sealing 
perspeetive on the strueture of intelleet. In P. L. 
Aekerman, P. C. Kyllonen, & R. D. Roberts (Eds.), 
Learning and individual differences: Process, trait, and 
content determinant (pp. 187 - 204). Washington, D.C.: 
APA Books. 

Ding, C. (2003). Exploratory longitudinal profde analysis via 
multidimensional sealing. Practical Assessment, 
Research & Evaluation, On-line serial 8. 

Ding, C. S., Davison, M. L., & Petersen, A. C. (2005). 
Multidimensional sealing analysis of growth and ehange. 
Journal of Educational Measurement, 42, 171-191. 

du Toil, S. H. C., & Cudeek, R. (2001). The analysis of 
nonlinear random eoeffieient regression models with 
LISREL using eonstraints. In R. Cudeek, S. H. C. du 
Toil, & D. Sorbom, (Eds.), Structural equation 
modeling: Present and future (pp. 259-278). 

Lineolnwood, IL: Seientifie Software International. 

Enders, C. K., & Bandalos, D. L. (2001). The relative 
performanee of full information maximum likelihood 
estimation for missing data in struetural equation models. 
Structural Equation Modeling, 8, 430 - 457. 

Foorman, B., Franeis, D., Novy, D., & Liberman, D. (1991). 
How letter-sound instruetion mediates progress in first- 
grade reading and spelling. Journal of Educational 
Psychology, 83, 456-469. 

Gold, M, S., Bender, P. M., & Kim, K. H. (2003). A 
eomparison of maximum-likelihood and asymptotieally 
distribution-free methods of treating ineomplete 
nonnormal data. Structural Equation Modeling, 10, M - 
19. 

Grewal, R., Cote, J. A., & Baumgartner, H. (2004). 
Multieollinearity and Measurement Error in Struetural 
Equation Models: Implieations for Theory 

Testing, Marketing Science, 23, 519-529. 

Huttenloeher, J. E., Haight, W., Bryk, A. S., & Seltzer, M. 


274 



Practical Issues of Growth Modeling Techniques 


(1991). Early vocabulary growth: Relation to language 
input and gender. Developmental Psychology, 27, 236 - 
249. 

Jdreskog, K. G. (1969). A general approach to confirmatory 
maximum likelihood factor analysis. Psychometrika, 34, 
183-202. 

Kim, S., Frisby, C. L., & Davison, M. L. (2004). Estimating 
cognitive profiles using profile analysis via 
multidimensional scaling (PAMS). Multivariate 
Behavioral Research, 39(4), 595 - 624. 

Kline, R. B. (1998). Principles and Practice of structural 
equation modeling. New York: The Guliford press. 

Kmenta, J. (1971). Elements of econometrics. New York: 
MacMillan. 

Kruskal, J. B. (1964). Nonmetric multidimensional scaling: A 
numerical method. Psychometrika, 29, 1 15 - 129. 

Linda, N. Y., Lee, S.-Y., & Poon, W.-Y. (1993). Covariance 
structure analysis with three level data. Computational 
Statistics & Data Analysis, 15, 159-178. 

Longford, N. T., & Muth'en, B. O. (1992). Factor analysis for 
clustered observations. Psychometrika, 57, 581-597. 

MacCallum, R. (1986). Specification searches in covariance 
structure modeling. Psychological Bulletin, 100, 107- 
120 . 

Meredith, W., & Tisak, J. (1990). Latent curve analysis. 
Psychometrika, 55, 107-122. 

Micceri, T. (1989). The unicorn, the normal curve, and other 
improbable creatures. Psychological Bulletin, 105, 156 - 
166. 

Muthen, B. O. (1989). Latent variable modeling in 
heterogeneous populations. ftycAometn'ka, 54, 557-585. 

Muthen, B. O. (1991). Analysis of longitudinal data using 
latent variable models with varying parameters. In L. M. 
Collins, & J. L. Horn (Eds.), Best methods for the 
analysis of change: Recent advances, unanswered 
questions, future directions. Washington, D.C.:APA. 

Muthen, B. O. (1994). Multilevel covariance structure 
analysis. Sociological Methods and Research, 22, 376- 
398. 


Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear 
models: Applications and data 

analysis methods. California: Saga Publications, Inc. 

Rogosa, D. R., & Willett, J. B. (1985). Understanding 
correlates of change by modeling individual differences 
in growth. Psychometrika, 50, 203-228 

Rovine, M. J. & Molenaar P. M. (1998). A LISREL model for 
the analysis of repeated measures with a patterned 
covariance matrix. Structural Equation Modeling, 5, 318 
-343. 

Smith, K. E., Landry, S. H., & Swank, P. R. (2000). Does the 
content of mothers’ verbal stimulation explain 
differences in the children’s development of verbal and 
nonverbal cognitive skills? Journal of School 
Psychology, 38, 27-49 

Stoolmiller, M. (1995). Using latent growth models to sutyd 
developmental processes. In J. M. Gottman (Ed), The 
analysis of change (pp. 103-138). Mahwah, NJ: Erlbaum. 

Vasu, E. S., & Elmore, P. B. (1975). The effect of 
multicollinearity and the violation of the assumption of 
normality on the testing of hypotheses in regression 
analysis. Paper presented at the Annual Meeting of the 
American Educational Research Association. Washington, 
D.C. 

Willett, J. B., & Sayer, A. G. (1994). Using covariance 
structure analysis to detect correlated and predictors of 
individual change over time. Psychological Bulletin, 116, 
363-381. 


Received March 9, 2007 
Revision received April 10, 2007 
Accepted June 26, 2007 


275 



